Daniel Kahneman–Gary Klein adversarial collaboration

The Daniel Kahneman–Gary Klein adversarial collaboration was a sustained attempt to turn an intellectual conflict into a shared investigation. Its decisive achievement was to distinguish genuine intuitive expertise from the illusion of validity: confidence alone cannot establish that an intuition is sound, but intuition can be trusted when a regular environment supplies learnable patterns and a professional has had prolonged practice and meaningful feedback.

Origins in Disagreement

Daniel Kahneman had long regarded academic disputes with suspicion. In his account, journal exchanges that begin with criticism often deteriorate into defensive replies and rejoinders, with little admission of error and little instruction for readers. Rather than continue that pattern, he developed the practice of adversarial collaboration: scholars who disagree about the science agree to write jointly about their differences and, at times, conduct research together. In particularly tense cases, an arbiter moderates the work.

Gary Klein represented a sharply contrasting intellectual tradition. As the leader of scholars and practitioners associated with Naturalistic Decision Making, he studied professionals making consequential decisions in real organizations. The group rejected what it saw as the excessive emphasis on failure in the heuristics-and-biases approach, distrusted artificial experiments that displaced real-world expertise, and remained skeptical that rigid algorithms should replace human judgment. Paul Meehl, an important figure for Kahneman’s criticism of clinical prediction, was not among their heroes.

The opposition was therefore substantive rather than merely personal. Kahneman’s thinking had been shaped by observing the illusion of validity in himself and by studying demonstrations that clinical prediction could be inferior to simpler methods. Klein’s views had been formed through field studies of fireground commanders, whose decisions often appeared impressively good under pressure. Yet Kahneman had never believed that intuition was invariably misguided. He admired Klein’s early work on firefighters and his analysis of how experienced professionals acquire intuitive skills, which made collaboration possible despite the absence of easy agreement.

The Question They Pursued

Kahneman invited Klein to help map the boundary separating the marvels of intuition from its flaws. Their specific question was whether, and under what conditions, an experienced professional who claimed to have an intuition could be trusted. The asymmetry between them was clear from the beginning: Klein would be more disposed to trust experienced judgment, while Kahneman would be more skeptical. The project’s value lay precisely in forcing those default attitudes into explicit comparison rather than allowing either one to remain an unexamined allegiance.

Their work lasted seven or eight years. They held many discussions, resolved numerous disagreements, nearly allowed the project to collapse more than once, and produced repeated drafts. The process changed the relationship itself: they became friends and ultimately published a joint article titled “Conditions for Intuitive Expertise: A Failure to Disagree.” The title captures the collaboration’s irony. They did not encounter genuine issues on which they disagreed, but neither did they simply converge on an undifferentiated belief in intuition. Their agreement concerned the conditions under which intuition works, not a blanket endorsement of intuitive judgment.

Two Models of Intuition

Klein’s fireground research supplied the strongest case for expert intuition. He and his collaborators initially investigated whether firefighting commanders made decisions by comparing a small set of alternatives. The hypothesis was wrong. Commanders usually generated only one plausible option, drew it from a repertoire of patterns accumulated through more than a decade of real and virtual experience, and then mentally simulated that option to determine whether it would work.

The process was neither purely automatic nor purely analytical. System 1, the fast and associative mode of thought, produced a tentative plan through pattern recognition. System 2, the slower and more effortful mode, tested the plan through mental simulation. If the action appeared appropriate, the commander implemented it. If it had shortcomings, the commander modified it; if it could not be repaired easily, the commander moved to the next plausible option and repeated the process. Klein developed this account into the recognition-primed decision model, which applies not only to firefighters but also to other domains of expertise, including chess.

This model complemented rather than contradicted the broader account of System 1 and System 2. System 1 operates automatically and quickly, often supplying impressions, intentions, feelings, and candidate actions. System 2 normally functions in a comparatively comfortable low-effort mode, endorsing many of System 1’s suggestions, but it is mobilized when a problem is difficult, surprising, or prone to error. In the collaboration’s view, skilled intuition is not a mystical alternative to reasoning: it is rapid recognition supported by experience and followed, when necessary, by deliberate checking.

Kahneman’s contribution was to show why the same subjective experience can be deceptive. When a difficult question receives no expert solution, the mind may substitute an easier one without noticing. A person asked whether to invest in Ford stock may instead answer whether the person likes Ford cars; an evaluation of a company’s commercial future may be dominated by impressions of its executives’ energy and competence. System 1 produces an answer quickly, and System 2 may endorse it because the answer is coherent and effortless, even though it addresses a different question.

Conditions for Trust

The collaboration’s central conclusion was that intuitive validity depends on the environment as much as on the individual. First, the environment must be sufficiently regular to contain patterns that can be learned. Second, the professional must have an opportunity to discover those regularities through prolonged practice. When both conditions are present, associative memory can recognize meaningful cues and generate quick, accurate predictions or decisions.

Fireground command provides the kind of setting in which these conditions may be met: situations recur in recognizable forms, professionals accumulate extensive experience, and actions produce consequences that can be examined. Chess offers another example. Acquiring chess expertise is slower than learning to read because the game has a far larger alphabet and its meaningful combinations contain many more elements. After thousands of hours of practice, however, chess masters can read a position at a glance; the few moves that come to mind are usually strong and may be creative, even when the precise position is unfamiliar.

The same professional may possess real skill in one task and lack it in another. A psychotherapist may develop good intuitions about what a patient will say next while having no comparable basis for predicting how well that patient will do a year later. A financial expert may be competent in several aspects of finance without being able to pick stocks, and an expert on the Middle East may know its history and politics without being able to forecast its future. The collaboration therefore treats expertise as task-specific, not as a transferable aura surrounding a person.

Regularity alone is not enough. A talented young chess player can make mistakes with great confidence while still developing toward mastery. Evaluation must ask whether the professional had an adequate opportunity to learn the relevant cues, even where the environment is structured. The crucial evidence is the provenance of the intuition—where it was formed, what feedback shaped it, and whether the cues were genuinely predictive—not the compelling quality of the intuition when it arrives.

Confidence, Prediction, and the Boundary of Expertise

Kahneman and Klein agreed that subjective confidence is not a reliable guide to intuitive validity. Confidence often arises from cognitive ease and coherence: a story comes readily to mind, contains no obvious contradiction, and lacks a competing scenario. But the associative machinery also suppresses doubt and retrieves information compatible with the dominant story. A person can therefore feel certain because the explanation is smooth, not because it is true. The practical rule that follows is severe: no one, including oneself, should be trusted to report how much trust their own judgment deserves.

Research on political and economic forecasting illustrates the danger. Philip Tetlock interviewed 284 people who earned their living commenting on or advising about political and economic trends, collecting more than 80,000 probability judgments about events such as coups, wars, and emerging markets. The experts performed worse than a simple equal-probability allocation across the possible outcomes. Even in regions they knew best, they were not significantly better than nonspecialists. Greater knowledge produced only a slight improvement, while the most knowledgeable and famous forecasters often developed a stronger illusion of skill and greater overconfidence.

The same pattern appeared among corporate financial officers. In a Duke University survey, officers of large corporations made 11,600 forecasts of the following year’s Standard & Poor’s index. The correlation between their estimates and the actual value was slightly below zero: when they predicted a decline, the market was slightly more likely than not to rise. The disturbing feature was not only the forecasts’ uselessness but the officers’ apparent inability to recognize it. Their 80 percent confidence intervals contained far too many surprises—67 percent of outcomes fell outside the expected boundaries, more than three times the anticipated rate.

Calibration studies provide another way to expose overconfidence. A properly calibrated judge’s stated probability intervals should contain the true values at the corresponding rates; for example, the true values should fall within a 98 percent interval on 98 percent of problems. In practice, actual values often fall outside the stated extreme bounds on about 30 percent of problems, revealing intervals that are systematically too narrow. Such overconfidence appears in both naive and sophisticated judges and is not eliminated simply by offering proper scoring rules. Anchoring helps explain the effect: people begin with a best estimate, adjust upward or downward insufficiently, and then mistake the resulting narrow range for justified certainty.

The collaboration thus separates two kinds of fast judgment. In a regular environment, prolonged practice can make pattern recognition reliable, and System 2 can test the recognized plan. In a low-validity environment, however, System 1 may answer a difficult question through substitution, creating coherence where no dependable pattern exists. The resulting intuition may feel indistinguishable from a valid one because both arrive quickly and confidently. The difference must be established by examining the environment, the learning history, the feedback, and the task’s actual predictability—not by listening to confidence alone.